Aspects of the present invention relate generally to the field of image processing, and more specifically to noise reduction.
In image processing systems, video or image data may be captured by a camera having a sensor. Conventional cameras convert the captured analog information from the sensor to digital data that is passed to an ISP for signal processing. The processed signal is then passed to a CPU or GPU for additional processing including filtering, encoding, image recognition, pattern or shape recognition, color enhancement, sharpening, or other image enhancing processes.
As well as capturing the desired image, the sensor may capture some background noise in the analog signal. The signal-to-noise ratio (SNR) for a captured signal reflects a comparison of the desired signal to the noise in the signal. As noise in a signal may cause distortion and visible artifacts in the captured image, SNR is also a reflection of the quality of the captured signal. The SNR of a captured image may vary based on the lighting available when capturing the image. In bright conditions with a lot of light, the SNR may be high. However, with dim or low lighting conditions, the captured image may have more noise and therefore a smaller SNR.
Conventional image processing systems use a CPU or GPU to filter the processed images to improve the quality of the signal and prepare the processed image for display by reducing noise and improving SNR. The CPU or GPU may implement a spatial or temporal filter for noise reduction. Spatial filters reduce or filter noise in a single captured frame by averaging the signals representing the captured frame to cut off outlying signals that likely represent noise. Spatial filters are conventionally implemented with a low pass filter. Temporal filters reduce or filter noise across multiple frames representing images captured over time. Then, for frames depicting the same scene, the difference between the frames may represent noise. The temporal filter identifies these differences and reduces the identified noise throughout the sequence of frames. Then the SNR will improve over time as the spatial filter uses the scene history to improve filtering operations.
However, temporal filtering may result in ghosting. Ghosting occurs when part of a first image appears in a second, later image. Strong temporal filtering, where the filtering relies in large part on the history of a sequence of images, may cause certain areas of a later image to be inappropriately filtered, resulting in noticeable ghosting and artifacts. This is particularly common when an object in a first image has moved in the second image or a change in lighting conditions results in an improved SNR. A noticeable artifact in an image may be the result of strong temporal based filtering causing ghosting, may be the result of motion blur, or may just be distortion caused by noise.
Accordingly, there is a need in the art to adapt to changing lighting and image conditions while optimizing the filtering operations to limit visible noise in the image.